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Documenting the Value of Non-Degree Credentials: 
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Data Exchange 


The Multistate Longitudinal Data Exchange (MLDE) facilitates data sharing between states from K-12 
education, higher education, and labor agencies. Its goal is to provide practitioners, policymakers, and 
researchers with a comprehensive data source to understand educational and career trajectories, including 
how these trajectories can cross State lines, to improve policies and programs serving students and provide 
better consumer information. The exchange, begun as a pilot in 2010, is a collaboration between the 
Western Interstate Commission for Higher Education (WICHE) and state agencies that house education 


and work data in multiple states and has been largely funded by the Bill & Melinda Gates Foundation. 
The Education & Employment Research Center (EERC) at Rutgers, The State University of New Jersey, has 
conducted research and evaluation on the inaugural years of the MLDE. This EERC brief is one of a series’ 
that explores the development of the MLDE and details the lessons learned about building and using 
longitudinal multistate data systems for policy and practice. 


In the light of the changing nature of work and live, the fields or skills covered by the credentials they 
education, state leaders across the nation are earned, the quality of the NDCs conferred, or the 
considering the role of non-degree credentials value of any given NDC to the individual who earned 
(NDCs) in postsecondary education. In fact, many it.4 While the need for good data on NDCs is clear, 
states have included non-degree credentials - credit systems for collecting and analyzing them are neither 
and noncredit certificates, industry certifications, well developed nor utilized. Some states have begun 
licensures, and apprenticeships - in their state’s to use State Longitudinal Data Systems (SLDS) to 
postsecondary goals.? Non-degree credentials are gather and aggregate data on NDCs.° Other states 
also at the forefront of states’ discussions about are employing existent data to better understand 
reducing equity gaps in education and the labor the value or quality of NDCs, thereby helping both 
market, filling labor market demands, and preparing policymakers and consumers to make more informed 
their workforces for the future. Essential to these decisions about the use of state and personal 
conversations is good data on the value of NDCs in resources.® 
the labor market and their potential with respect to These activities are all important steps but remain 
educational and career pathways. limited by state lines, providing only partial answers 
Current data indicate that non-degree credentials about national patterns and possible directions for 
have become more prevalent over the last 15 years, policy and program activities. Another factor limiting 
with more than one-quarter of Americans now such analyses is the rise of nontraditional private 
holding an NDC.3 However, those data are not robust providers, such as bootcamps and other for-profit 
in terms of who those Americans are or where they opportunities, which have become an increasing 

oe RUTGERS WIE J”. — 

LONGITUDINAL School of Management fi 

DATA EXCHANGE and Labor Relations Western Interstate Commission 


for Higher Education 


@ @ e Education and Employment Research Center 


MLDE Issue Brief /2 


source of NDCs. Few of these private providers 
collect the student data (particularly Social Security 
numbers) necessary for researchers, state agencies, 
or the providers themselves to evaluate how those 
who participate in these opportunities fare in the 
workplace. Today, to truly understand the value and 
potential of NDCs, we need multistate outcomes data 
that recognize the realities of migration in and out 

of states - of individuals crossing state borders to 
study, to work, to deal with family circumstances, or 
for other reasons. We also need information about 
how the different types of educational opportunities 
that are becoming more prevalent in the modern 

age can benefit students and workers while helping 
provide the skilled talent today’s economy requires. 
The Multistate Longitudinal Data Exchange (MLDE) is 
a nascent system that has the potential to respond to 
these needs. 


The MLDE was piloted in 2010 and expanded 

shortly after as a collaboration between the Western 
Interstate Commission for Higher Education (WICHE) 
and agencies within participating states that collect 
data related to education and employment (e.g., K-12 
and higher education, workforce agencies). The MLDE 
facilitates the exchange of data between states to 
provide practitioners, policymakers, and researchers 
with a comprehensive data system that can enhance 
their understanding of 21st-century educational and 
career pipelines and trajectories. The MLDE is well 
suited to provide information on NDCs because it can 
document the movement of people who hold or are 
in pursuit of NDCs both within and across state lines 
as they move between education and careers. This 
brief explores the various ways that multistate data 
and the MLDE might be able to help states better 
answer many lingering questions about NDC policy 
and practice. 


Over the past five years, the MLDE's third-party 
evaluator, the Education & Employment Research 
Center (EERC) at Rutgers, the State University of New 
Jersey, has engaged in a mixed-methods research 
evaluation of the MLDE implementation process. This 
brief draws on EERC’s qualitative research, which 
included 40 interviews with state leaders and WICHE 
staff, observations of user group meetings, surveys, 
and MLDE document analysis. 


In the following sections, we present the reasons why 


states have joined the MLDE or considered joining the 
LDE. We then discuss states’ interest in data and the 
ways data can be used to evaluate NDCs. We include 
in our discussion ideas heard from state policymakers 
during our study as well as ideas found in current 
iterature and media on the topic. We conclude with 
the recommendation that as states and territories 
continue to develop and build their own longitudinal 
data systems, they should consider the value of both 
adding data on NDCs and sharing that data with other 
States. 


How might cross-state data sharing 
improve policymakers’, practitioners’, 
and researchers’ understanding of non- 
degree credentials? 


Over the past five years, while studying the 
implementation of the MLDE, EERC has accumulated 
information on both the need for and potential uses 
of multistate longitudinal education and workforce 
data. Although EERC's interviews with state leaders did 
not specifically focus on NDCs, many of the leaders’ 
comments about the need for more and better 
data, and how they would use such data if it were 
available, can be interpreted to refer at least in part 
to the relative lack of data on NDCs. NDC data are 
therefore a natural extension of the MLDE's current 
infrastructure that could yield rich results. 


The addition of NDC data to state agency datasets 
and SLDS is an important step in the pursuit of 
answers about the prevalence and value of NDCs. 

A recent report by the Workforce Data Quality 
Campaign (WDQC) looked at the extent to which 
states are already collecting NDC data. Although just 
beginning, infrastructures are being developed for 
higher quality data: 36 states reported collecting data 
on most for-credit certificates from their public two- 
year institutions; 27 states reported collecting data 
about most registered apprenticeship certificates; 
and 22 states reported having access to most 
licensing data.? However, WDOQC and others have 
identified a number of significant issues with respect 
to collecting and aggregating NDC data. For example, 
it is often unclear what forms of credentials qualify 
as an NDC in a given dataset. Are all types of NDCs 
receiving equal attention by data collectors, or is 
there a privileging of some over others? Naturally, 


Value of Non-Degree Credentials 


MLDE Issue Brief /3 


NDCs from postsecondary institutions and providers 
that participate in workforce training programs - 
when they are already incorporated into state data 
infrastructures - are most likely to be linked to 
earnings information, but states that rely on those 
data sources could be missing information about a 
large swath of NDCs that may (or may not) have value 
in the workforce. Nonetheless, the WDQC report 
showcased growing awareness of the need for good 
data on NDCs and the progress that has already been 
made in securing that information. 


In addition to collecting data, some states have begun 
to use available data to implement quality-assurance 
processes that examine either the content of the 
credential or its outcomes, typically measured in 
terms of educational or labor market achievements.® 
In fact, 30 states have already developed or are in the 
process of developing a list of “credentials of value” to 
help guide the allocation of public dollars for financial 
aid, workforce development, or other programs.? In 
addition, some states are making data available to 
the public to assist consumers with educational and 
career decision making. These important efforts will 
benefit if more states join the MLDE and actively share 
their data on NDCs."° 


Further, standardization of analysis of shared data 
sets would help answer questions about the pathways 
of state-to-state migrants and the future of work, and 
it would expand the value of existent SLDS for state 
policymakers and their constituents. 


The Pathways of State-to-State Migrants 


One of the most prominent reasons state leaders 
gave for wanting to collect and exchange data with 
other states was to enhance their understanding 

of the movement of workers and students: those 
who migrate in and out of the state, those who 
commute daily out of state for school or work. 
Tracking the movement of students and workers 
would enable enhanced analysis of the alignment of 
state educational investments. It would also allow the 
exploration of questions regarding the extent to which 
the state draws in migrants from other states vs. the 
extent to which its own students, once they complete 
their education or training, move out of state for 
better job opportunities. Analysis of education and 
employment pipelines could also inform discussions 


of human capital and the “balance of trade” between 
states,"’ thereby informing state policy and practice 
decisions with respect to educational resources and 
workforce development. Data show that just over 2 
percent of the U.S. population moved to a new state 
during the period of 2016 through 2018.'? An analysis 
of cross-state commuting data from 2011 shows that 
an additional 4 percent of workers crossed state lines 
for employment as commuters, with some states 
posting substantially higher numbers." 


Significance for NDCs: Multistate data on NDCs 
could help states track long-term educational 
and labor market outcomes of individuals who 
have earned one or more NDCs and who have 
migrated into or out of a state. Such tracking 
could enhance understanding of the effects 

of NDCs on educational and career pathways 
and financial well-being. These data could help 
inform decisions regarding which NDCs should 
be supported and promoted by shedding light 
on the relative quality and value of NDCs to the 
people who earn them. 


Future of Work 


Some of the state leaders with whom EERC spoke 
reflected on the changing value of the MLDE for 
their state. Many commented on the impact of 
the global economy on job opportunities and how 
new technologies were transforming the nature of 
work. These changes required more real-time data 
to facilitate decision making about educational and 
economic policies and programs. 


Significance for NDCs: NDCs are frequently 
mentioned in the context of the future of work, 
where they are most often described as valuable 
tools for training and retraining workers so 
they have the requisite knowledge and skills 
needed for new technologies. Tracking the 
pathways of NDC seekers and holders could 
help policymakers respond more strategically 
to the changing needs of industry sectors 

and employers and better prepare the next 
generation of workers. Workers, as well as 
employers, will continue to cross state borders, 
including some who will work remotely in states 
other than the one in which they live. Multistate 
data will therefore become increasingly useful 
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to regional planners and state policymakers 
involved with the allocation of economic 
resources, including tax incentives, as well 
as educators as they develop or refine their 
curriculum and program offerings. 


Expanding the Value of SLDS Data for States and 
Constituents 


Some state leaders talked about using the MLDE 

to provide more comprehensive data to their 
constituents, thereby fostering more informed 
decision making. As one respondent said, “/ think 
that's what ultimately our goal is: to be transparent and 
to provide data that is useful to the public, not just state 
agencies.” 


Significance for NDCs: Improving the data 
available to constituents aligns well with the 
work cited above and the fact that some 30 
states have already developed lists of NDC 
“credentials of value.” It also aligns well and 
would build on work currently being performed 
by the Credential Engine to document and count 
credentials.'* Access to multistate data through 
the MLDE would enable a more comprehensive 
analysis of the impact of NDCs on educational 
and labor market outcomes. This would help 
individuals, including incumbent workers, to 
make more informed choices about pursuing 
specific NDCs or other academic credentials. 

It would also help industry, unions, and 
professional organizations make more informed 
decisions about the viability of NDCs and their 
provision of such credentials. 


Filling Other Data Gaps on NDCs 


An increasing number of NDCs come from private 
providers who may be under no mandate to report 
on their students. These credentials include formal 
certifications in computer coding through private 
bootcamps, digital badges provided by employers, 
and numerous other types of training and education. 
While some of these programs may look and feel 
like formal education programs, they can exist in 

a regulatory gray area where they claim not to be 
subject to state laws on the provision of education 
programs and avoid reporting requirements (if 
any) that would allow policymakers and the public 


to evaluate how the credential they offer benefits 
participants in the workplace. 


Even states that regulate these providers may not 
require them to submit accountability or evaluation 
data to state data systems; states that do not 
regulate them certainly have no such requirements. 
Additionally, providers of many of these newer 
credentials are unlikely to collect or share Social 
Security numbers, which are usually a key data 
element required to access earnings information. 


For providers that have a strong interest in 
understanding more about how those that earn 
their credentials fare in the workforce, an approach 
that borrows from the MLDE has the potential to 
help them get the information they need. The data 
infrastructure supporting the MLDE was designed 
to provide a state with earnings information about 
its students in cases where that state did not have 
students’ Social Security numbers, but other states 
did. To grossly simplify the approach, the system 
allowed states without Social Security numbers to 
request that other states provide those numbers to 
participating earnings data repositories. 


The MLDE approach mimics efforts from Idaho, 
where the state obtained earnings data on high 
school graduates who did not enter the state's 
postsecondary system (and thus the state never had 
access to the students’ Social Security numbers)."5 


Conceptually, the solution is straightforward: 


e Develop partnerships between NDC providers 
and the sources of the desired outcomes data 
as well as the state’s higher education data 
repository. This allows the training provider to 
submit individual-level data to the state’s higher 
education data repository, which can then use 
this information to match with its own records. 


e Have the postsecondary education data 
source request earnings data from the state's 
Unemployment Insurance system and return 
aggregated data to the provider, while carrying 
out any needed state research or analysis on its 
own. 


e Use the data and information for evaluation of 
education and training programs or consumer 
information. 
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In many cases, the state agency holding earnings 
data for evaluation purposes also has the 
postsecondary data that is necessary in the matching 
phase, therefore significantly easing the burden of 
developing the necessary partnerships. The analysis 
f these data points would provide information on 
how the participants in these new NDC programs fare 
in the workforce. As a proof-of-concept, responsible 
nd ethical providers would most likely be interested 
in participating on a voluntary basis in exchange 

for data that could help significantly improve their 
programs. 


Oo 
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Although such tracking and analysis would be limited 
to only those individuals who had participated in a 
states’ formal public postsecondary system, it would 
provide a starting point for evaluation purposes. 
States participating in multistate data initiatives like 
the MLDE could broaden the scope of such an effort 
to include data from across the country. 


Conclusion 


As NDCs become more prominent in today's labor 
market and educational landscape, data on their 
outcomes will be increasingly important for the public 
sector, industry, and workers as they make decisions 
about preparing the workforce and distributing 
resources. While some states have begun to look 

at NDC data using their SLDS, to truly understand 

the value and potential of non-degree credentials 

we need multistate outcomes on a variety of NDCs. 
Students and workers migrate into and out of states 
for a variety of reasons, and it is important that these 
realities are captured in data systems and analyses 
where possible. This may be even more true now with 
the education and employment impacts of COVID-19. 
With the future of work being so uncertain, and with 
budgets being severely constrained by the pandemic, 
it will be crucial for states to better understand the 
outcomes of their investments. Understanding the 
quality and value of NDCs in educational and career 
pathways requires cross-state data access now more 
than ever. 
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For further information about the MLDE, please contact Patrick 
Lane at plane@wiche.edu, or visit the project website: https://www. 
wiche.edu/key-initiatives/multistate-longitudinal-data-exchange/, 
The five briefs in this series include: Building a Multistate Governance 
System; Building Trust for Inter-Organizational Data Sharing: The 

Case of the MLDE; Designing the Architecture of a Multistate Data 
Sharing Model: Diffusion of an Innovation: Lessons from the Multistate 
Longitudinal Data Exchange; and Documenting the Value of Non- 
Degree Credentials: The Potential Role of the Multistate Longitudinal 
Data Exchange. 


For further information about evaluation research in data sharing 

or workforce development, please contact Heather McKay, director 
of Rutgers’ Education & Employment Research Center, at hmckay@ 
rutgers.edu, or visit the EERC website: http://smlr.rutgers.edu/eerc. 
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